# Efficient Attention Mechanism

Provide the AttnGate weights for the decoding phase in the SeerAttention-R paper, supporting the inference tasks of the Qwen3-4B model

QA model fine-tuned from ModernBERT-base-nli, supporting long-context processing

Mistral-7B-Instruct-v0.2 is an enhanced instruction fine-tuned large language model based on Mistral-7B-Instruct-v0.1, achieving 30% sparsity through Wanda pruning method without requiring retraining while maintaining competitive performance.

Long-sequence Nyströmformer model trained on WikiText-103 v1 dataset, supports sequence processing up to 4096 tokens

Nystromformer model trained on the WikiText-103 dataset, supporting long sequence processing (2048 tokens)

LongT5 is a text-to-text transformation model based on the T5 architecture, employing transient global attention mechanism for efficient processing of long sequence inputs

This model is a distilled version of the Data-efficient image Transformer (DeiT), pretrained and fine-tuned on ImageNet-1k at 224x224 resolution, efficiently learning from a teacher model through distillation.

This is a pruned and optimized BART-base model, specifically fine-tuned on the CNN/DailyMail dataset for summarization tasks.

This is a Chinese pre-trained model based on the BigBird architecture, optimized for Chinese text processing and supporting long text sequence handling.